Is the U.S. government designating too many documents as 'classified'?

The mishandling of classified documents continues to make headlines, first with the FBI's Aug. 2022 raid of former President Trump's Mar-a-Lago home and, more recently, with the revelation that classified documents had been found at President Biden's private office and also in his home. Historian Matthew Connelly says one reason we see problems like this is that far too many government records are being categorized as "classified."

On average, Connelly says, records are marked as classified three times every second, generating so many secret documents that it's practically impossible to preserve them all.

"More and more of what's classified are things like PowerPoint presentations and spreadsheets and text messages and video conferences," he says. "The sheer volume is something we can't even measure anymore in paper. ... We don't know what's stored in the cloud — or, in some cases, deleted and just destroyed completely so that no one will ever know."

Connelly says that over the last 20 years, he's noticed that documents he would have expected to find in the archives are simply not there. "If you want to try to do the history of the 1980s or even the 1970s, you find that there are just huge gaps in the documentary record," he says. "And I couldn't help but ask myself: How much more out there is missing? What is it that we don't know?"

In the past, classified records were archived, preserved and eventually declassified — allowing researchers and casual history buffs to pore over them for new details about events in the past, such as the Cuban Missile Crisis or the civil rights movement. But in his new book, The Declassification Engine: What History Reveals About America's Top Secrets, Connelly makes the case that, unless the process is reformed, huge numbers of documents will never be reviewed for declassification.

"Nobody's ever going to be allowed to see any of [the documents now being labeled classified] unless some other official decides that information is safe to release to the public," he says. But the method for reviewing these records and releasing them hasn't changed in 80 years — despite the fact that the volume of data has changed; it's still a requirement, Connelly says, that officials review "every one of these pages, page by page."

If technology is part of the problem, it could also be part of the solution. Connelly and some data scientists have used artificial intelligence and machine learning to develop a system to analyze huge troves of records to determine which should be truly classified and which can be made public.

"With computers, we're generating more information all the time, including more and more classified information," he says. "We have to devise technology that's going to allow us to prioritize that information that really does require protection and then accelerate the release of everything else."

Interview highlights

On how so many documents have become classified

Documents are not supposed to be classified as national security information unless it poses a threat to American national security. And over the years, practically every president has issued an executive order where they try to define what it means for something to be "top secret" or "secret" or "confidential" or what have you.

Now, the problem is, of course, that there are 2,000-3,000 [appointed] officials who have the power to classify something new, a new program, a new technology. They have the power to decide that everything related to that program is going to be classified at a certain level. Let's say it's "top secret." But once they've done that, every official who's involved — and there are literally millions of people who have security clearances — every one of them ... is required to stamp anything related to that as being classified at the same level. And so what happens is, maybe to start with some of this stuff is sensitive, but over time so much information starts to be classified that it can be ludicrous. And some good examples of that include when, if you recall during the so-called scandal over Hillary Clinton's emails, a lot of what ended up getting classified was, for example, newspaper reporting about drone strikes. So even information that's out there in the public, even things in some cases that were headline news at the time. Once they're exchanged among senior officials, then even that information ends up getting classified.

On Obama classifying more documents than any previous president

To the extent we can actually measure, like how many times government officials create new secrets, that reached its all-time peak under the Obama administration. And there are other ways you could look at this as well. Under the Obama administration, the Justice Department indicted more people under the Espionage Act for disclosing classified information than all previous administrations combined. And, in fact, the amount of spending on government secrecy even reached new heights, to the point where by the end of his administration, the U.S. government is spending some $18 billion on protecting national security information.

On President Trump's approach to classification

Here's a president who said that when he became president, he was going to, for example, release all the secrets that have been held up to that point about the Kennedy assassination. But somehow, in the end, he decided not to. And in fact, by the end of his administration, he was saying that everything he touched, everything, was classified at the highest level.

As much as Donald Trump seemed to represent a dramatic departure from traditions of the American presidency, in one way he was entirely consistent with every president who came before him. And that's because he decided that he didn't even need to write a new executive order for secrecy. Every president going all the way back to FDR had written an executive order that was supposed to define and control national security information. But Donald Trump decided that he was happy with the executive order written by Barack Obama — and so he just kept everything in place.

On Trump claiming he had personally declassified documents found at Mar-a-Lago

Presidents do have the sovereign power of secrecy. But there's no evidence that he actually did. I mean, he suggested he could have done it in his own mind. But actually this system is incredibly complex. Above all, it's about paper. So every decision like this about whether something gets declassified, or how it's classified, all of this has to be recorded in paper. And so there's no evidence that Trump declassified any of these records.

On classified emails found on Hillary Clinton's personal server

Some of those records were classified at the highest level. Some of them involve what's called "sensitive compartmentalized information." That might have been information related to signals intelligence [information gained by intercepting communications]. It will likely be a long time before we know what actually was in there. And certainly there's good reason to believe that some of what was in there was actually pretty innocuous and it was being withheld largely for political reasons, diplomatic reasons, in the sense that these were things that the U.S. government didn't want to disclose officially, even if they were already known or largely known to the public. What bothers me more, as a historian, even as a citizen, is how many of those emails were destroyed, because before Clinton handed over those 30,000 or so emails that she decided were public records, there [were] tens of thousands of other records that she and her lawyers decided were private. They deleted them so that no one will ever know what it is that she decided no one else was going to be allowed to see.

On classified records left at an office and a home by then-Vice President Biden

I think you can look at it two ways: One way of looking at it is that this is just more evidence about how state secrecy is out of control. They just can't keep track of all the secrets that they're generating, because there are just too many of them. And so even if you credit Joseph Biden and the people around him and you think that they were responsible [stewards of these documents], then you still have to ask yourself: How is it that they lost track of records that, apparently, at least in some cases, were classified as top secret?

But the other way to look at this is how it is that ... you see that the people, whether it was the president or, in this case, the vice president, the people who are most invested in state secrecy are the ones who are most determined to cling to these secrets.

On developing an algorithm that can predict what materials should be classified vs. declassified

We found hundreds of examples of things that were completely innocuous, like hotel reservations that were classified at the time as "Secret." On the other hand, we found things that the algorithm predicted should have been classified as "Secret," but weren't. ... We found things that almost certainly should have been classified.

I actually had the good fortune to have colleagues who had security clearances or at least did in the past and had a lot of experience in this area. And I gave them a blind test. I asked them to look at those documents that were classified and those documents that were unclassified. And I asked them to tell us which of those records should have been classified and at what level. And time and again they were astonished. They were more likely to agree with the algorithm than to agree with the officials who originally classified this information or didn't classify it.

So the conclusion I took away from this is that there is a lot of human error in official secrecy. There are many examples. We found many hundreds of examples where officials either were over-classifying things or examples where they were under-classifying things, things that could actually have been potentially harmful for national security.

Sam Briger and Joel Wolfram produced and edited this interview for broadcast. Bridget Bentz, Molly Seavy-Nesper and Meghan Sullivan adapted it for the web.